Method for depth lookup table signaling

ABSTRACT

A method and apparatus for depth lookup table (DLT) signaling in a three-dimensional and multi-view coding system. The method identifies one or more pictures to be processed. If one or more pictures contain depth data, then the method determines the DLT associated with said one or more pictures, applies predictive coding to the DLT based on the previous DLT, includes syntax related to the DLT in the PPS, and includes first bit-depth information related to first depth samples of the DLT in the PPS. The first bit-depth information is consistent with second bit depth information signaled in a sequence level. The method further signals the PPS in a video bitstream for a sequence including said one or more pictures. A circuit is also provided that embodies circuitry configured to carry out the operations specified above.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/123,882, filed Sep. 6, 2016, which is a national phaseapplication of PCT patent application no. PCT/CN2015/074391, filed Mar.17, 2015, which is a continuation-In-Part of PCT Patent Application,Ser. No. PCT/CN2014/073611, filed on Mar. 18, 2014. The priorityapplications are all hereby incorporated by reference in theirentireties.

TECHNICAL FIELD

The present invention relates to three-dimensional and multi-view videocoding. In particular, the present invention relates to method for DepthLookup Table signaling.

BACKGROUND

Three-dimensional (3D) television has been a technology trend in recentyears that is targeted to bring viewers sensational viewing experience.Multi-view video is a technique to capture and render 3D video. Themulti-view video is typically created by capturing a scene usingmultiple cameras simultaneously, where the multiple cameras are properlylocated so that each camera captures the scene from one viewpoint. Themulti-view video with a large number of video sequences associated withthe views represents a massive amount data. Accordingly, the multi-viewvideo will require a large storage space to store and/or a highbandwidth to transmit. Therefore, multi-view video coding techniqueshave been developed in the field to reduce the required storage spaceand the transmission bandwidth. A straightforward approach may simplyapply conventional video coding techniques to each single-view videosequence independently and disregard any correlation among differentviews. Such straightforward techniques would result in poor codingperformance. In order to improve multi-view video coding efficiency,multi-view video coding always exploits inter-view redundancy. Thedisparity between two views is caused by the locations and angles of thetwo respective cameras.

Depth lookup table (DLT) has been adopted into 3D-HEVC. Very often,there are only limited values appearing in the depth component.Therefore, DLT is a compact representation of the valid values in ablock. When a CU is coded in Intra simplified depth coding (SDC) mode ordepth map modeling (DMM) mode, DLT is used to map the valid depth valuesto DLT indexes. FIG. 1 demonstrates an example of DLT representation ofdepth values in a picture. While the range of depth values is from 0 to255, only 5 depth values (i.e., 50, 108, 110, 112 and 200) appear in thepicture. Accordingly, the DLT consists of 5 values with indexes from 0to 4. The DLT is signaled in the picture parameter set (PPS) and it isup to the encoder to generate the DLT.

In the current 3D-HEVC (Three-Dimensional Coding Based on HighEfficiency Video Coding) draft standard, DLT is signaled as an extensionto picture parameter set (PPS). The syntax elements related to DLTsignaling are described in the following tables.

As shown in Table 1, the DLT parameter information, pps_dlt_parameters() is incorporated in the PPS if the PPS extension flag,pps_extension_type_flag[0] is asserted.

TABLE 1 Descriptor pic_parameter_set_rbsp( ) {  ... u(1) pps_extension_flag u(1)  if( pps_extension_flag ) {   for ( i = 0; i <8; i++ )    pps_extension_type_flag[ i ] u(1)   if(pps_extension_type_flag[ 0 ] )    poc_reset_info_present_flag u(1)   if(pps_extension_type_flag[ 2 ] )    pps_dlt_parameters( ) u(1)   if(pps_extension_type_flag[ 7 ] )    while( more_rbsp_data( ) )    pps_extension_data_flag u(1)  }  rbsp_trailing_bits( ) }

The syntax structure for the DLT parameter information,pps_dlt_parameters( ) is shown in Table 2. When the flag,dlt_present_flag has a value of 1, information related to DLT isincorporated in pps_dlt_parameters( ) as shown in Table 2. Theinter-view DLT prediction flag, inter_view_dlt_pred_enable_flag[i] equalto 1 indicates that the i-th depth lookup table is predicted from thedepth lookup table of the 0-th depth lookup table. On the other hand,the flag, inter_view_dlt_pred_enable_flag[i] equal to 0 indicates thei-th depth lookup table is not predicted from any other depth lookuptable.

TABLE 2 Descriptor Note pps_dlt_parameters( ) {  dlt_present_flag u(1) if( dlt_present_flag) {   pps_depth_layers_minus1 u(6)  pps_bit_depth_for_depth_views_minus8 u(4)   for( i=0; i <=pps_depth_layers_minus1; i++) {    dlt_flag[ i ] u(1)    if( dlt_flag[ i] ) {     inter_view_dlt_pred_enable_flag[ i ] u(1)     if(!inter_view_dlt_pred_enable_flag[ i ] )      dlt_bit_map_rep_flag[ i ]u(1)     if( dlt_bit_map_rep_flag[ i ] )      for( j = 0; j <=depthMaxValue; j++ )       dlt_bit_map_flag[ i ][ j ] u(1)     else     entry_table( i )    }   }  } }

The DLT signaling according to the existing 3D-HEVC standard has someissues. First, inter-view prediction can be applied to DLT as indicatedby a corresponding flag, inter_view_dlt_pred_enable_flag[i]. If thisflag is set to 1, the i-th DLT is predicted from the 0-th DLT. On theother hand, dlt_flag[i] indicates whether the i-th DLT exists.Therefore, if dlt_flag[0] is 0 and inter_view_dlt_pred_enable_flag[i]with i>0 is 1, the i-th DLT is predicted from a non-existed DLT.

In the existing 3D-HEVC standard, pps_bit_depth_for_depth_views_minus8is signaled to indicate the bit-depth for samples of the depth componentin the picture. However, the bit-depth for samples in the depthcomponent is also signaled in the sequence level as indicated bybit_depth_luma_minus8 which is signaled in sequence parameter set (SPS).Therefore. There is potential contradiction between these two syntaxelements if these two syntax elements are different.

In the existing 3D-HEVC standard, the DLT is signaled in all PPSs of thebit-stream including the texture video in all views since the flagdlt_present_flag can be set to 1 for the texture data. However, the DLTis only required by the depth component of each view. According to theexisting 3D-HEVC standard, the DLT is signaled in four PPSs in totalwhen there are 3 views. As shown in FIG. 2, three PPSs are sent for thetexture component for three views, (i.e., T0, T1 and T2) individually.On the other hand, only one PPS is used and shared by the depthcomponent for all three views (i.e., D0, D1 and D2). Therefore, 4 copiesof the same content of DLT will be signaled in the 4 PPSs repeatedlywith an obvious redundancy.

It is desirable to develop methods to overcome these issues withoutcausing noticeable impact on the performance.

SUMMARY

A method and apparatus for depth lookup table (DLT) signaling in athree-dimensional and multi-view coding system are disclosed. A methodand apparatus for depth lookup table (DLT) signaling in athree-dimensional and multi-view coding system. The method identifiesone or more pictures to be processed. If one or more pictures containdepth data, then the method determines the DLT associated with said oneor more pictures, applies predictive coding to the DLT based on theprevious DLT, includes syntax related to the DLT in the PPS, andincludes first bit-depth information related to first depth samples ofthe DLT in the PPS. The first bit-depth information is consistent withsecond bit depth information signaled in a sequence level. The methodfurther signals the PPS in a video bitstream for a sequence includingsaid one or more pictures.

In another embodiment of the application, a circuit is also providedthat embodies circuitry configured to carry out the operations specifiedabove.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 illustrates an example of depth lookup table (DLT), where thetable consists of five entries corresponding to 50, 108, 110, 112 and200 indicated by indexes 0 through 4.

FIG. 2 illustrates an example of depth lookup table (DLT) signaling inthe picture parameter set (PPS) for the texture component and the depthcomponent according to conventional method.

FIG. 3 illustrates an example of depth lookup table (DLT) signaling inthe picture parameter set (PPS) for the texture component and the depthcomponent according to an embodiment of the present invention.

FIG. 4 illustrates another example of depth lookup table (DLT) signalingin the picture parameter set (PPS) for the texture component and thedepth component according to an embodiment of the present invention.

FIG. 5 illustrates an exemplary flowchart of depth lookup table (DLT)signaling for a 3D or multi-view coding system according to anembodiment of the present invention.

DETAILED DESCRIPTION

As mentioned before, there are various issues with the depth lookuptable (DLT) signaling in the existing High Efficiency Video Coding(HEVC) based 3D video coding. Accordingly, embodiments of the presentinvention are disclosed to overcome these issues. The followingdescription is of the best-contemplated mode of carrying out theinvention. This description is made for the purpose of illustrating thegeneral principles of the invention and should not be taken in alimiting sense. The scope of the invention is best determined byreference to the appended claims.

One aspect of the present invention addresses validity of DLTprediction. As shown in the PPS DLT parameter syntax of the existing3D-HEVC, predictive DLT coding is allowed for all depth layersregardless of whether the depending DLT exists or not. In oneembodiment, whether a corresponding DLT exists for predicting a currentDLT is checked first. If the corresponding DLT exists, predictive DLTcoding is allowed for the current DLT to use the corresponding DLT as apredictor. If the corresponding DLT does not exist, predictive DLTcoding is not applied to the current DLT regardless of whether theinter-view DLT prediction is enabled or not as indicated by aninter-view DLT prediction enable flag,inter_view_dlt_pred_enable_flag[i]. Alternatively, if the correspondingDLT required for predicting the i-th DLT does not exist, the flaginter_view_dlt_pred_enable_flag[i] is forced to be 0. In anotherembodiment, the flag inter_view_dlt_pred_enable_flag[i] is inferred as 0if the DLT for predicting the i-th DLT does not exist.

An exemplary syntax table to incorporating an embodiment of the presentinvention is shown in Table 3. For the i-th DLT, the flaginter_view_dlt_pred_enable_flag[i] is incorporated only when the 0-thDLT exists, i.e., dlt_flag[0] being 1.

TABLE 3 if( dlt_flag[ i ] ) {  if(dlt_flag[0])  inter_view_dlt_pred_enable_flag[ i ] u(1)

When the flag inter_view_dlt_pred_enable_flag[i] is 1 and the DLT thatis originally used to predict the i-th DLT does not exist, the predictorfor the i-th DLT can be changed to another DLT that exists. Instead ofchanging to an existing DLT, a predefined DLT can be used in this caseas well. For example, the predefined DLT may include all possiblevalues, such as 0, 1, . . . , 255, in the depth component. In anotherexample, the predefined DLT contains no values.

Another aspect of the present invention addresses the consistency of bitdepth information in different layers of the syntax. For example, theconsistency of the bit-depth indication can be checked for the sequencelevel. To be specific, all bit-depth indications for depth data signaledin a video sequence must be the same as the bit-depth indicationsignaled in the sequence parameter set (SPS). Also, the PPS level bitdepth indication (i.e., pps_bit_depth_for_depth_views_minus8) is set tobe the same as the SPS level bit depth indication (i.e.,bit_depth_luma_minus8).

In another embodiment, the bit depth consistency is achieved by scalingwhen the bit depth indications are different in different levels. Forexample, depth values of the DLT signaled in the PPS can be scaled ifthe PPS level bit depth indication, pps_bit_depth_for_depth_views_minus8and the SPS level bit depth indication, bit_depth_luma_minus8 aredifferent. For example, a depth value D of the DLT can be scaledaccording toD′=(D+offset)>>(pps_bit_depth_for_depth_views_minus8−bit_depth_luma_minus8)if pps_bit_depth_for_depth_views_minus8 is greater thanbit_depth_luma_minus8. In another example, the scaling can be doneaccording toD′=D<<(bit_depth_luma_minus8−pps_bit_depth_for_depth_views_minus8) ifpps_bit_depth_for_depth_views_minus8 is lower thanbit_depth_luma_minus8. The offset can be any integer such as 0 or(1<<(pps_bit_depth_for_depth_views_minus8−bit_depth_luma_minus8−1)).

Another aspect of the present invention addresses redundancy in the DLTsignaling. Since the DLT is not needed by the texture data, DLT is notsignaled in the PPS for the texture component. In other words, DLT isnot signaled in the PPS for the texture only layer. On the other hand,the DLT for the depth data of all views can be signaled in a single PPSthat is shared by depth components of all views as shown in FIG. 3.Alternatively, the DLT for the depth data of each view can be signaledin an individual PPS that is used by the particular depth component ofthe corresponding view as shown in FIG. 4.

In one embodiment, one PPS may only signal the DLT for the depthcomponent associated with one view only. In other words, one PPS onlysignal the DLT for one layer. In another embodiment, a slice may use theDLT signaled in the PPS that contains PPS identification,pps_pic_parameter_set_id with the same value as the sliceidentification, slice_pic_parameter_set_id in the slice header for thisslice.

In one embodiment, the DLT signaled in one PPS, noted as P1, can bepredicted by a DLT signaled in a different PPS, noted as P0.Furthermore, the pps_pic_parameter_set_id of P0 can be signaled in P1 tolocate the PPS (i.e., P0) containing a DLT to be used as a predictor forthe DLT in PPS P1.

FIG. 5 illustrates an exemplary flowchart of depth lookup table (DLT)signaling for a 3D or multi-view coding system according to anembodiment of the present invention. The system identifies one or morepictures to be processed as shown in step 510. Whether said one or morepictures contain only texture data is checked in step 520. If the resultis “Yes”, the process goes to step 570. If the result is “No”, theprocess goes to step 530. In step 530, the DLT associated with said oneor more pictures is determined. If a previous DLT required forpredicting the DLT exists, predictive coding is applied to the DLT basedon the previous DLT as shown in step 540 and syntax related to the DLTis included in the PPS as shown in step 550. Furthermore, firstbit-depth information related to first depth samples of the DLT isincluded in the PPS as shown in step 560. The first bit-depthinformation is consistent with second bit-depth information signaled ina sequence level for second depth samples of a sequence containing saidone or more pictures. The PPS is then signaled in a video bitstream fora sequence including said one or more pictures as shown in step 570.

The flowchart shown above is intended to illustrate an example of3D/multi-view coding using DLT signaling in three-dimensional andmulti-view coding according to an embodiment of the present invention. Aperson skilled in the art may modify each step, re-arranges the steps,split a step, or combine steps to practice the present invention withoutdeparting from the spirit of the present invention.

The above description is presented to enable a person of ordinary skillin the art to practice the present invention as provided in the contextof a particular application and its requirement. Various modificationsto the described embodiments will be apparent to those with skill in theart, and the general principles defined herein may be applied to otherembodiments. Therefore, the present invention is not intended to belimited to the particular embodiments shown and described, but is to beaccorded the widest scope consistent with the principles and novelfeatures herein disclosed. In the above detailed description, variousspecific details are illustrated in order to provide a thoroughunderstanding of the present invention. Nevertheless, it will beunderstood by those skilled in the art that the present invention may bepracticed.

Embodiment of the present invention as described above may beimplemented in various hardware, software codes, or a combination ofboth. For example, an embodiment of the present invention may correspondto one or more electronic circuits integrated into a video compressionchip or program code integrated into video compression software toperform the processing described herein. An embodiment of the presentinvention may also be program code to be executed on a Digital SignalProcessor (DSP) to perform the processing described herein. Theinvention may also involve a number of functions to be performed by acomputer processor, a digital signal processor, a microprocessor, orfield programmable gate array (FPGA). These processors can be configuredto perform particular tasks according to the invention, by executingmachine-readable software code or firmware code that defines theparticular methods embodied by the invention. The software code orfirmware code may be developed in different programming languages anddifferent formats or styles. The software code may also be compiled fordifferent target platforms. However, different code formats, styles andlanguages of software codes and other means of configuring code toperform the tasks in accordance with the invention will not depart fromthe spirit and scope of the invention.

The invention may be embodied in other specific forms without departingfrom its spirit or essential characteristics. The described examples areto be considered in all respects only as illustrative and notrestrictive. The scope of the invention is therefore, indicated by theappended claims rather than by the foregoing description. All changeswhich come within the meaning and range of equivalency of the claims areto be embraced within their scope.

The invention claimed is:
 1. A method of depth coding using a depthlookup table (DLT) in a three-dimensional and multi-view coding system,the method comprising: identifying one or more pictures to be processed;if said one or more pictures contain depth data: determining the DLTassociated with said one or more pictures; applying predictive coding tothe DLT based on the previous DLT; including syntax related to the DLTin the PPS; and including first bit-depth information related to firstdepth samples of the DLT in the PPS, wherein the first bit-depthinformation included in the PPS is consistent with second bit depthinformation signaled in a sequence level; and signaling the PPS in avideo bitstream for a sequence including said one or more pictures. 2.The method of claim 1, wherein the second bit depth information signaledin the sequence level corresponds to luma samples.
 3. The method ofclaim 1, wherein the second bit depth information signaled in thesequence level is bit_depth_luma_minus8.
 4. The method of claim 1,wherein the second bit depth information signaled in the sequence levelis for second depth luma samples of a sequence containing said one ormore pictures.
 5. The method of claim 1, wherein first bit depth valueindicated by the first bit depth information is the same as second bitdepth value indicated by the second bit-depth information.
 6. Anapparatus for depth coding using a depth lookup table (DLT) in athree-dimensional and multi-view coding system, the apparatus comprisingone or more electronic circuits configured to: identifying one or morepictures to be processed; if said one or more pictures contain depthdata: determining the DLT associated with said one or more pictures;applying predictive coding to the DLT based on the previous DLT;including syntax related to the DLT in the PPS; and including firstbit-depth information related to first depth samples of the DLT in thePPS, wherein the first bit-depth information included in the PPS isconsistent with second bit depth information signaled in a sequencelevel; and signaling the PPS in a video bitstream for a sequenceincluding said one or more pictures.